Pre-trained LMs have had a significant impact on Natural Language Processing (NLP), with the "pre-train and fine-tune" paradigm rapidly becoming the predominant approach to apply effective models on a wide variety of downstream tasks [1-3, inter alia]. However, one of the main concerns when working with LMs is the paucity of annotated data, especially for specific domains or low-resource languages, required to fine-tune the additional classification layer on top of these models for downstream tasks, such as classification. Recently, prompt-based tuning has started to affirm as a promising way to perform similar tasks, significantly reducing the need for annotated data. This approach has been proven to be very effective with Large Language Models (LLMs) [4]. However, it is often the case that LLMs are not available for low-resource languages, and that their performance drastically decreases when they are challenged on specific domains. Moreover, in the Digital Transformation era, businesses frequently need to integrate artificial intelligence systems into their application ecosystems. This requires them to utilize specialized, publicly available models while also employing effective methods to leverage these models in scenarios where annotated language resources are unavailable, thereby operating in a zero-shot mode. Hence, we decided to evaluate two smaller domain-specific encoder models: BureauBERTo [5], a LM further pre-trained on Italian bureaucratic texts (i.e., administrative acts, banking and insurance documents), and Italian Legal BERT [6] (henceforth referred to as Ita-Legal-BERT), a LM adapted to the Italian legal domain, on various classification tasks on domain-specific data exploiting a prompt-based technique in a zero-shot scenario. Additionally, we compared the performance of both models with that of a generic Italian model, UmBERTo.

Prompt learning's fine-tune performance on text classification tasks has attracted the NLP community. This paper applies it to resume information extraction, improving existing methods for this task. We created manual templates and verbalizers tailored to resume texts and compared the performance of Masked Language Model (MLM) and Seq2Seq PLMs. Also, we enhanced the verbalizer design for Knowledgeable Prompt-tuning, contributing to prompt template design across NLP tasks. We present the Manual Knowledgeable Verbalizer (MKV), a rule for constructing verbalizers for specific applications. Our tests show that MKV rules yield more effective, robust templates and verbalizers than existing methods. Our MKV approach resolved sample imbalance, surpassing current automatic prompt methods. This study underscores the value of tailored prompt learning for resume extraction, stressing the importance of custom-designed templates and verbalizers.